Autonomous driving requires high accuracy, real-time localization of vehicles. Currently, most vehicle navigation has been accomplished using a global positioning system (GPS), which provides a real-time location with a 95% confidence interval of 7.8 meters, according to the US government. However, in complicated urban environments, reflection in GPS signals can further increase this error, such that a determined location may be off by as much as 30 meters. Given that the width of many lanes is only 3 to 4 meters, this accuracy is not sufficient to properly localize an autonomous vehicle so that it can make safe route planning decisions. Other sensors, such as inertial measurement units (IMUs) can increase the accuracy of localization by taking into account vehicle movement, but these sensors tend to drift and still do not provide sufficient accuracy for localization.
Generally, image-based vehicle localization uses feature-matching using, for example, GPS data to determine a possible vehicle location on a map. The map provides a set of stable features to which features present in an image may be matched. For example, features in the image may be matched to features in the map over different locations on the map to perform an accurate localization of the vehicle. However, these feature matching techniques generally fail to provide a metric indicating how well an image feature matches a map feature (i.e., to quantify a feature error in terms of a pose error).